Determining whether to include or exclude device data for determining a network communication configuration for a target device

ABSTRACT

Techniques for determining whether to include or exclude device data for determining a network communication configuration for a target device are disclosed. Device data for devices in a communication network are monitored for a particular time period. A particular set of device data, for a particular device, indicates a change in network communication behavior of the particular device. Responsive to determining that there was no event that may have caused the change in behavior, the particular set of device data is not used for determining a configuration for a target device. Responsive to determining that there was an event that may have caused the change in behavior, the event type of the event is determined. Whether to include or exclude the particular set of device data, for determining a configuration for a target device, is determined based on the event type.

RELATED APPLICATIONS; INCORPORATION BY REFERENCE

This application is related to U.S. patent application Ser. No. 15/428,470, filed Feb. 9, 2017, where is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to network communications. In particular, the present disclosure relates to determining whether to include or exclude device data for determining a network communication configuration for a target device.

BACKGROUND

A communication network provides communication links among a set of nodes. Examples of communication networks include the Internet, an intranet, a public network, a private network, a wide area network, a local area network, an ad hoc network, mesh network, a peer-to-peer network, a distributed network, and an Internet of Things (IoT) network. The communication links may be implemented using a variety of wireless and/or wired communication technologies. Transmission media include but are not limited to electrical cables, coaxial cables, twisted pair wires, optical fibers, radio waves, and/or microwaves. Embodiments and examples described in this patent application may be used in one or more types of communications networks.

In recent years, Wireless Local Area Network (WLAN) technologies have emerged as a fast-growing market. Among the various WLAN technologies, the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard is the dominating technology and is frequently used for WLANs. The IEEE 802.11 standard includes operations in one or more bands (such as the 2.400-2.500 GHz band and/or the 4.915-5.825 GHz band), and multiple channels within each band. Wi-Fi is an example of a communication technology based on the IEEE 802.11 standard. Other examples of wireless communications technologies include WiGig, Zigbee, and Bluetooth.

Nodes in a communications network include client devices and network devices. A network device is configured to provide, control, facilitate and/or manage the communication links in the communications network. A network device may provide a communication link between a client device and a communications network. The network device and/or client device may initiate the establishment of the communication link. As an example, the client device may transmit a request to connect to the communication network. As another example, the network device may transmit a broadcast message to the client device, even if the client device had not requested access to the communications network. Examples of network devices include an access point (AP), a cable modem, a cellular tower, a router, a switch, and a bridge. An AP that communicates wirelessly with client devices may be referred to as a “wireless AP.”

The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and they mean at least one. In the drawings:

FIG. 1 illustrates an example of a system for determining a network communication configuration for a target device based on a target device profile, in accordance with one or more embodiments;

FIG. 2 illustrates an example of a supervised machine learning system for obtaining a target device profile, in accordance with one or more embodiments;

FIG. 3 illustrates an example of an unsupervised machine learning system for obtaining a target device profile, in accordance with one or more embodiments;

FIG. 4 illustrates an example of a profile building system for obtaining a target device profile, in accordance with one or more embodiments;

FIG. 5 illustrates an example of an anomaly detection system for obtaining a target device profile, in accordance with one or more embodiments;

FIG. 6 illustrates an example set of operations for obtaining a target device profile, in accordance with one or more embodiments;

FIG. 7 illustrates an example set of operations for determining whether to include or exclude device data for determining a network communication configuration for a target device, in accordance with one or more embodiments; and

FIG. 8 shows a block diagram that illustrates a computer system in accordance with one or more embodiments.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding. One or more embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in a different embodiment. In some examples, well-known structures and devices are described with reference to a block diagram form in order to avoid unnecessarily obscuring the present invention.

-   1. GENERAL OVERVIEW -   2. PROFILE GENERATION SYSTEM ARCHITECTURE -   3. OBTAINING A TARGET DEVICE PROFILE -   4. DETERMINING WHETHER TO INCLUDE OR EXCLUDE DEVICE DATA FOR     CONFIGURING A TARGET DEVICE -   5. EXAMPLE EMBODIMENT -   6. MISCELLANEOUS; EXTENSIONS -   7. HARDWARE OVERVIEW

1. General Overview

One or more embodiments include determining whether to include or exclude device data for determining a network communication configuration for a target device. A device monitoring engine monitors and collects device data for devices in a communications network. An anomaly detection engine determines whether there is a change in network communication behavior for one or more devices. The anomaly detection engine determines whether any events, that may have caused the change in behavior, have occurred. The anomaly detection engine determines an event type of the event that may have caused the change in behavior. Based on the event type, the anomaly detection engine determines whether the device data, reflecting the change in behavior, should be used in determining a configuration for the target device. Examples of event types include (a) events associated with a change in a target device, (b) events associated with a change in a wireless environment of the target device, (c) events associated with a change in the communications network of the target device, (d) events associated with manual settings, and (e) seasonal events. In the above examples, event types (a)-(c) would indicate that the event has caused a permanent change in the behavior of the devices. The anomaly detection engine determines that the device data should be used for determining a configuration for the target device. Event types (d)-(e) would indicate that the change in behavior may be a temporary anomaly. The anomaly detection engine refrains from using the device data for determining a configuration for the target device, at least until the anomaly detection engine obtains additional information indicating that the change in behavior is permanent.

The device monitoring engine continues collecting device data for the devices in the communications network. The anomaly detection engine determines whether the change in network communication behavior is observed for a minimum time period. If the change in network communication behavior is observed for at least the minimum time period, then the anomaly detection engine determines that the change in behavior is permanent. The anomaly detection engine determines that the device data should be used for determining a configuration for the target device. Additionally or alternatively, the anomaly detection engine determines whether the change in network communication behavior is observed for multiple devices. If the number of devices exhibiting the change in behavior is greater than a specified threshold, then the anomaly detection engine determines that the change in behavior is permanent. The anomaly detection engine determines that the device data should be used for determining a configuration for the target device. Otherwise, the anomaly detection engine refrains from using the device data reflecting the change in behavior.

One or more embodiments described in this Specification and/or recited in the claims may not be included in this General Overview section.

2. Profile Generation System Architecture

FIG. 1 illustrates an example of a system for determining a network communication configuration for a target device based on a target device profile, in accordance with one or more embodiments. FIG. 1 includes a target device profile generation framework 102, a target device profile 104, a configuration engine 108, and a network communication configuration 112 for a target device. In one or more embodiments, the system for determining a network communication configuration for a target device may include more or fewer components than the components illustrated in FIG. 1. The components illustrated in FIG. 1 may be local to or remote from each other. The components illustrated in FIG. 1 may be implemented in software and/or hardware. Each component may be distributed over multiple applications and/or machines. Multiple components may be combined into one application and/or machine. Operations described with respect to one component may instead be performed by another component. Components that are labeled with a same number across FIGS. 1-5 refer to the same component.

In one or more embodiments, a target device profile 104 is a profile for a particular client device (also referred to as a “target device”). A target device profile 104 includes expected behaviors 106 for the target device. The expected behaviors 106 may be determined based on actual behaviors of the target device and/or other client devices. Types of behaviors for a client device include but are not limited to:

(a) protocol behavior;

(b) connection behavior;

(c) power-save behavior;

(d) roaming behavior;

(e) response to a particular configuration algorithm; and

(f) response to a particular radio-frequency (RF) environment.

Protocol behavior includes behaviors associated with a network communication protocol. Examples include operations that are performed by the target device to associate with an AP; the sequence of the operations that are performed by the target device to associate with an AP; the duration of time used in completing each operation performed by the target device to associate with an AP; the type and/or size of frame aggregation used by the target device; the frequency at which the target device performs scanning; and the triggers that cause the target device to perform scanning.

Connection behavior includes behaviors associated with associating with an AP. Examples include what band the target device uses to connect with an AP; what conditions cause the target device to connect using a particular band rather than another band; the duration of time used by the target device to find available APs; and the duration of time used by the target device to determine which AP to connect with.

Power-save behavior includes behaviors associated with a power-save mode of the target device. Examples include the frequency with which the target device enters power-save mode; the triggers that cause the target device to enter power-save mode; the triggers that cause the target device to exit power-save mode; and any behaviors of the target device while in power-save mode.

Roaming behavior includes behaviors associated with roaming. Examples include the triggers that cause the target device to roam; the duration of time used in roaming from one AP to another AP; the amount of data that is lost during roaming; and the duration of time used in determining whether to roam.

Response to a particular configuration algorithm includes behaviors of the target device in response to a configuration algorithm. As further described below, examples of configuration algorithms include a “intelligent client steering algorithm” and a “sticky client management algorithm.” An intelligent client steering algorithm (also referred to as a “steering algorithm”) is configured to steer a client device from one radio to another radio if the switch results in a performance improvement that is above a threshold. Various steering methods may be used, such as, transmitting a de-authentication message to the client device; instructing the client device to move to another radio using an 802.11v message; and modifying another protocol message, format, or contents. Examples of responses to a steering algorithm include the duration of time used to connect with the particular AP identified by the steering algorithm; and whether the target device correctly connected to the particular AP identified by the steering algorithm. Examples of responses to a sticky client management algorithm include whether the target device correctly disconnected from an AP as determined by the sticky client management algorithm; and whether the target device successfully connected to another AP after disconnecting from a particular AP, as determined by the sticky client management algorithm.

Response to a particular RF environment includes behaviors of the target device in response to one or more attributes of an RF environment. Attributes of an RF environment include a noise level; an interference level; a number of wireless devices operating within a wireless range of the target device; a number of APs within a wireless range of the target device; and a number of channels utilized by the APs within a wireless range of the target device.

In one or more embodiments, a network communication configuration 112 for a target device specifies one or more settings for the target device, other client devices in the communications network, and/or network devices in the communication network. A network communication configuration 112 may specify different settings for different time periods (such as, time of day and/or day of week).

Examples of settings for a network device (such as an AP) include conditions for accepting or rejection a connection request from the target device; a RF channel utilized by the network device; a transmit power; a Modulation and Coding Scheme (MCS) or a Physical Layer (PHY) rate; a probe response threshold; a disassociate threshold; a receive sensitivity; a receive gain; a transmit cell size; a receive cell size; the operations and sequence thereof for associating with a client device; a threshold time period for performing a particular operation before generating a time-out error.

Examples of settings for a client device (such as the target device) include which AP to attempt a connection with; a RF channel utilized by the client device; a transmit power; a MCS or PHY rate; a signal-strength threshold for roaming; a receive sensitivity; a receive gain; a transmit cell size; a receive cell size; the operations and sequence thereof for associating with a network device; a threshold time period for performing a particular operation before generating a time-out error; a retry rate; and an aggregation size.

A network device and/or a client device is implemented on one or more digital devices. The term “digital device” generally refers to any hardware device that includes a processor. A digital device may refer to a physical device executing an application or a virtual machine. Examples of digital devices include a computer, a tablet, a laptop, a desktop, a netbook, a server, a web server, a network policy server, a proxy server, a generic machine, a function-specific hardware device, a hardware router, a hardware switch, a hardware firewall, a hardware firewall, a hardware network address translator (NAT), a hardware load balancer, a mainframe, a television, a content receiver, a set-top box, a printer, a mobile handset, a smartphone, a personal digital assistant (PDA), a wireless receiver and/or transmitter, a base station, a communication management device, a router, a switch, a controller, an access point, and/or a client device.

In one or more embodiments, a target device profile generation framework 102 is configured to generate a target device profile 104. Components associated with the target device profile generation framework 102 are further described below with reference to FIGS. 2-5. Examples of operations for generating a target device profile 104 are described below with reference to FIGS. 6-7.

In one or more embodiments, a configuration engine 108 is configured to determine a network communication configuration 112 for a target device based on a target device profile 104. The configuration engine 108 determines the network communication configuration 112 by applying one or more configuration algorithms 110 to the target device profile 104. As an example, a target device profile may indicate that a target device is expected to use 20.0 ms to perform a particular association operation under a specific network configuration, RF environment, time-of-day, and/or other conditions. A configuration algorithm may indicate that the maximum time period to wait for completion of the particular association operation is determined by adding 5.0 ms to the expected time period necessary for completing the particular association operation. A configuration engine may apply the configuration algorithm to the expected behavior of the target device. The configuration engine may configure an AP, in the RF environment of the target device, to wait at least 25.0 ms before generating a time-out error during execution of the particular association operation or taking a specific corrective action to recover from or prevent a potential connectivity issue.

A target device profile generation framework 102 and/or a configuration engine 108 may be implemented or executed as part of an Expert system, Rule-Based system, Self-Learning system, or Machine-Learning system, which implements Machine Intelligence or Artificial Intelligence techniques. The target device profile generation framework 102 and/or the configuration engine 108 may be executed and/or distributed on one or more network devices that are local to a communication network provided by the network devices. Additionally or alternatively, the target device profile generation framework 102 and/or the configuration engine 108 may be executed and/or distributed on one or more digital devices that are remote from the communication network, such as a controller implemented in a cloud environment.

FIG. 2 illustrates an example of a supervised machine learning system for obtaining a target device profile, in accordance with one or more embodiments. FIG. 2 includes a labeled global dataset 202, a new global dataset 212, a supervised machine learning algorithm 214, a classifier function 216, classifications 218, and feedback 222. In one or more embodiments, the supervised machine learning system may include more or fewer components than the components illustrated in FIG. 2. The components illustrated in FIG. 2 may be local to or remote from each other. The components illustrated in FIG. 2 may be implemented in software and/or hardware. Each component may be distributed over multiple applications and/or machines. Multiple components may be combined into one application and/or machine. Operations described with respect to one component may instead be performed by another component. Components that are labeled with a same number across FIGS. 1-5 refer to the same component.

In one or more embodiments, a supervised machine learning algorithm 214 (also referred to as a “supervised learning algorithm”) is configured to determine a classifier function 216 based on a training dataset. An supervised machine learning algorithm 214 may be implemented using one or more algorithms well-known in the art, such as support vector machine (SVM), neural networks, pattern recognition, and/or Bayesian statistics.

In one or more embodiments, a classifier function 216 determines classifications 218 for one or more sets of device data. A classifier function 216 is applied to a set of device data to determine a particular class, of a candidate set of classes, for the set of device data. Each class includes device data associated with at least one common device attribute. In an embodiment, each class includes device data associated with the same device type. Examples of device types include Apple iPhone 7, Apple iPhone 6, and Samsung Galaxy S6. In other embodiments, each class includes device data associated with another common device attribute, such as the same manufacturer, the same operating channel, and/or the same multiple-input and multiple-output (MIMO) setting. As illustrated, a classifier function 216 divides a global dataset 212 into multiple device type datasets 220 a-b.

In one or more embodiments, a global dataset 212 includes multiple sets of device data 204 c-d. A particular set of device data (illustrated as device data 204 a, device data 204 b, device data 204 c, or device data 204 d) includes attributes and behaviors of a particular client device. For example, device data 204 a includes attributes 206 a and behaviors 208 a of one client device. Device data 204 b includes attributes and behaviors of another client device. Device data 204 c includes attributes and behaviors of another client device. Device data 204 d includes attributes and behaviors of another client device.

Examples of device attributes include a product type of the device (such as, whether the device is a laptop, desktop, or smartphone); a manufacturer of the device; a manufacturing location and/or date of the device; a device type of the device (such as, whether the device is an iPhone 7, iPhone 6, or Samsung Galaxy S6); an operating system and version thereof executing on the device; an application and version thereof executing on the device; an operating channel of the device; a MCS or PHY rate of the device; an aggregation size of the device; a user registered for and/or logged into the device; a geographical location of the device (such as, longitudinal and latitudinal coordinates of the device, or a particular room and/or building at which the device is located); a particular network to which the device is connected; a particular AP, cable modem, and/or cellular tower to which the device is connected; and a time of day at which the device is making a connection.

Examples of behaviors are discussed above with reference to expected behaviors 106 of FIG. 1.

In one or more embodiments, a device type dataset (such as, device type dataset 220 a or device type dataset 220 b) is a subset of the global dataset 212. A particular device type dataset includes sets of device data associated with the same device type.

In one or more embodiments, feedback 222 includes information indicating an actual classification for each set of device data. Feedback 222 may indicate whether the classifications 218, determined by the classifier function 216, are correct or not. Feedback 222 may be determined based on user input entered via a user interface. Additionally or alternatively, feedback 222 may be determined using another application.

As an example, a classifier function may determine that a particular set of device data corresponds to the device type, Apple iPhone 7. A user may review the particular set of device data and manually determine that the particular set of device data corresponds to Apple iPhone 6. The user may enter feedback via a user interface to indicate that the classification determined by the classifier function is incorrect. The user may enter feedback to indicate that the particular set of device data corresponds to iPhone6.

In one or more embodiments, a labeled global dataset 202 is a training dataset for input into a supervised machine learning algorithm 214. The labeled global dataset 202 includes multiple sets of device data 204 a-b, and labels 210 a-b corresponding to each set of device data 204 a-b. A label, corresponding to a set of device data, indicates an actual classification for the set of device data. The label may be generated based on feedback 222. As illustrated, device data 204 a is annotated with label 210 a. Device data 204 b is annotated with label 210 b.

Initially, the labeled global dataset 202 may be a bootstrap dataset that is used to initiate the supervised machine learning system. Subsequently, the labeled global dataset 202 may be expanded and/or modified based on device data collected for client devices in a communication network. The labeled global dataset 202 may be expanded and/or modified, in real-time, using device data collected for devices in a communication network as the device data is being collected.

As an example, initially, a labeled global dataset 202 may be input into a supervised machine learning algorithm 214. The supervised machine learning algorithm may determine a classifier function 216. A global dataset 212 may be newly collected for devices in a communication network. The classifier function 216 may be applied to the global dataset 212 to determine classifications 218. Feedback 222 may be obtained to add labels to the global dataset 212. The global dataset 212, with the added labels, may be added to the labeled global dataset 202. The expanded labeled global dataset 202 may be input into the supervised machine learning algorithm 214. The supervised machine learning algorithm may modify and/or refine the classifier function 216 based on the new device data of the global dataset 212. The labeled global dataset 202 is iteratively expanded and/or modified, such that the accuracy of the classifier function 216 is iteratively improved.

FIG. 3 illustrates an example of an unsupervised machine learning system for obtaining a target device profile, in accordance with one or more embodiments. FIG. 3 includes a global dataset 212, a classifier function 216, device type datasets 220 a-b, unsupervised machine learning algorithms 314 a-b, clusters 322 a-f. In one or more embodiments, the unsupervised machine learning system may include more or fewer components than the components illustrated in FIG. 3. The components illustrated in FIG. 3 may be local to or remote from each other. The components illustrated in FIG. 3 may be implemented in software and/or hardware. Each component may be distributed over multiple applications and/or machines. Multiple components may be combined into one application and/or machine. Operations described with respect to one component may instead be performed by another component. Components that are labeled with a same number across FIGS. 1-5 refer to the same component.

As described above with reference to FIG. 2, a global dataset 212 includes multiple sets of device data 204 c-d. As illustrated, device data 204 c includes attributes 206 c and behaviors 208 c of one client device. Device data 204 d includes attributes and behaviors of another client device.

The global dataset 212 is input into a classifier function 216. The classifier function 216 classifies the sets of device data 204 c-d based on a device type. The classifier function 216 divides the global dataset 212 into multiple device type datasets 220 a-b. As illustrated, device type dataset 220 a includes device data 204 c. Device type dataset 220 b includes device data 204 d.

In one or more embodiments, an unsupervised machine learning algorithm (such as, unsupervised machine learning algorithm 314 a or unsupervised machine learning algorithm 314 b) is configured to identify one or more clusters 322 a-f in a particular dataset. An unsupervised machine learning algorithm (also referred to as an “unsupervised learning algorithm”) may be implemented using one or more algorithms well-known in the art, such as clustering, and/or neural networks.

The unsupervised machine learning algorithm 314 a is applied to each of the device type datasets 220 a. The unsupervised machine learning algorithm 314 b is applied to the global dataset 212. The unsupervised machine learning algorithm 314 a and the unsupervised machine learning algorithm 314 b may be the same as or different from each other.

As illustrated, the application of the unsupervised machine learning algorithm 314 a to the device type dataset 220 a generates cluster group 320 a. Cluster group 320 a includes clusters 322 a-b. The application of the unsupervised machine learning algorithm 314 a to the device type dataset 220 b generates cluster group 320 b. Cluster group 320 b includes clusters 322 c-d. The application of the unsupervised machine learning algorithm 314 b to the global dataset 212 generates cluster group 320 c. Cluster group 320 c includes clusters 322 e-f.

In one or more embodiments, a cluster (such as, cluster 322 a, cluster 322 b, cluster 322 c, cluster 322 d, cluster 322 e, or cluster 322 f) is group of device data that exhibits a correlation between (a) one or more device attributes and (b) one or more behaviors. As illustrated, the cluster 322 a indicates a correlation between attributes 324 a and behaviors 326 a. The sets of device data in a particular cluster are more similar to each other than to sets of device data in other clusters.

Clusters 320 a-f may be stored in one or more data repositories. A data repository is any type of storage unit and/or device (e.g., a file system, database, collection of tables, or any other storage mechanism) for storing data. Further, a data repository may include multiple different storage units and/or devices. The multiple different storage units and/or devices may or may not be of the same type or located at the same physical site. Further, a data repository may be implemented or may execute on the same computing system as an unsupervised machine learning algorithm 314 a, unsupervised machine learning algorithm 314 b, and/or profile builder 406 (which is further described with reference to FIG. 4). Alternatively or additionally, a data repository may be implemented or executed on a computing system separate from an unsupervised machine learning algorithm 314 a, unsupervised machine learning algorithm 314 b, and/or profile builder 406. A data repository may be communicatively coupled to an unsupervised machine learning algorithm 314 a, unsupervised machine learning algorithm 314 b, and/or profile builder 406 via a direct connection or via a network.

FIG. 4 illustrates an example of a profile building system for obtaining a target device profile, in accordance with one or more embodiments. FIG. 4 includes cluster groups 320 a-c, a target device 402, a profile builder 406, and a target device profile 104. In one or more embodiments, the profile building system may include more or fewer components than the components illustrated in FIG. 4. The components illustrated in FIG. 4 may be local to or remote from each other. The components illustrated in FIG. 4 may be implemented in software and/or hardware. Each component may be distributed over multiple applications and/or machines. Multiple components may be combined into one application and/or machine. Operations described with respect to one component may instead be performed by another component. Components that are labeled with a same number across FIGS. 1-5 refer to the same component.

As described above, the application of an unsupervised machine learning algorithm to each of a device type dataset as well as a global dataset generates multiple cluster groups 320 a-c.

In one or more embodiments, a profile builder 406 refers to hardware and/or software configured to generate a target device profile 104, for a target device 402, using one or more clusters within cluster groups 320 a-c. The profile builder 406 generates the target device profile 104, including the expected behaviors 106 of the target device 402, based on attributes 404 of the target device 402. Examples of attributes 404 of a target device 402 are described above with reference to the device attributes (such as attributes 206 a) illustrated in FIG. 2. Examples of expected behaviors 106 of a target device 402 are described above with reference to FIG. 1. The profile builder 406 includes a cluster selection module 408, an aggregation module 410, and/or a merger module 412.

The cluster selection module 408 is configured to select clusters relevant to the target device 402 based on attributes 404. The cluster selection module 408 may select the relevant clusters in two phases. In the first phase, the cluster selection module 408 selects cluster groups 320 a-c that are relevant to the target device 402. In the second phase, the cluster selection module 408 selects clusters, within each selected cluster group, that are relevant to the target device 402. Examples of operations for selecting relevant clusters are described below with reference to FIG. 6.

The aggregation module 410 is configured to aggregate different behaviors associated with multiple relevant clusters to form a target device profile 104. As an example, attributes of a target device may include both an operating channel of 36 (at 40 MHz width) and a signal-to-noise ratio (SNR) of 20 dB. A first relevant cluster may indicate a correlation between (a) an operating channel of 36 (at 40 MHz width) and (b) a connection time of 30 ms. A second relevant cluster may indicate a correlation between (a) an SNR of 20 dB and (b) a roaming time of 40 ms. Since the first relevant cluster and the target device share a device attribute (that is, operating channel of 36), an aggregation module may include the behavior of the first relevant cluster (that is, connection time of 30 ms) into the target device profile. Since the second relevant cluster and the target device share a device attribute (that is, SNR of 20 dB), the aggregation module may include the behavior of the second relevant cluster (that is, the roaming time of 40 ms) into the target device profile. Hence, the aggregation module aggregates the behaviors of multiple relevant clusters to obtain the expected behaviors of the target device. The expected behaviors of the target device include: a connection time of 30 ms and a roaming time of 40 ms. Examples of operations for aggregating behaviors are described below with reference to FIG. 6.

The merger module 412 is configured to merge conflicting values, for the same behavior, associated with multiple relevant clusters to form a target device profile 104. As an example, attributes of a target device may include both an operating channel of 36 (at 40 MHz width) and an SNR of 20 dB. A first relevant cluster may indicate a correlation between (a) an operating channel of 36 (at 40 MHz width) and (b) a connection time of 30 ms. A second relevant cluster may indicate a correlation between (a) an SNR of 20 dB and (b) a connection time of 45 ms. Each of the first relevant cluster and the second relevant cluster share a device attribute with the target device. However, the first relevant cluster and the second relevant cluster are associated with different values for the same behavior (that is, the connection time). A merger module may determine a weighted average of the different values for the connection time in order to determine an expected value for the connection time for the target device. Examples of operations for merging values, for the same behavior, are described below with reference to FIG. 6. Further examples of operations for merging values, for the same behavior, are described in the related U.S. patent application Ser. No. 15/428,470, filed Feb. 9, 2017, which is hereby incorporated by reference in its entirety.

FIG. 5 illustrates an example of an anomaly detection system for obtaining a target device profile, in accordance with one or more embodiments. FIG. 5 includes a device monitoring engine 502, an event detection engine 506, an anomaly detection engine 510, an included dataset 514, and an excluded dataset 516. In one or more embodiments, the anomaly detection system may include more or fewer components than the components illustrated in FIG. 5. The components illustrated in FIG. 5 may be local to or remote from each other. The components illustrated in FIG. 5 may be implemented in software and/or hardware. Each component may be distributed over multiple applications and/or machines. Multiple components may be combined into one application and/or machine. Operations described with respect to one component may instead be performed by another component. Components that are labeled with a same number across FIGS. 1-5 refer to the same component.

In one or more embodiments, a device monitoring engine 502 is configured to monitor and collect device data 204 c-e for one or more devices in a communication network. The collected device data 204 c-e may be referred to as a “raw dataset 504.” In an embodiment, the device monitoring engine 502 is implemented on one or more APs and/or other network devices. As an example, a set of APs may monitor client devices in an RF environment. The APs may collect information about the attributes and behaviors of each client device. The APs may store the attributes and behaviors of each client device as a separate set of device data.

In an embodiment, device data 204 c-e within the raw dataset 504 may indicate a change in network communication behavior of one or more client devices in the communication network. A change in behavior refers to a change in value for a particular behavior, exhibited by a particular client device, that is greater than a threshold.

As an example, a device monitoring engine may monitor the connection time of a particular client device. Based on past device data for the particular client device, the average connection time may be 30 ms. However, the device monitoring engine may determine that the connection time of the particular client device is currently 50 ms. The change in value, between the historical average connection time and the current connection time, may be 20 ms. The change in value of 20 ms may be greater than a threshold. Hence, in this example, the device data for the particular client device indicates a change in behavior for the particular client device.

In one or more embodiments, an event detection engine 506 is configured to detect an occurrence of an event 508 that may have caused a change in network communication behavior of a particular client device. The event detection engine 506 may monitor events related to client devices and/or events related to a communications network.

Examples of events related to a client device include a change (such as, an update) in an operating system and/or application executing on the client device; a physical shutdown of the client device; a manual reset of the client device; a change in the user of the client device.

Examples of events related to a communications network include events related to an RF environment. Examples of events related to a communication network include a change in the deployment of network devices in the RF environment (such as the number of network devices, and/or the locations of network devices); a change in the configurations of network devices; a change in the noise level; a change in the interference level; and a change in the physical layout of the RF environment (such as, an additional table in the RF environment, and/or a removal of a particular wall in the RF environment).

In one or more embodiments, an anomaly detection engine 510 is configured to determine whether a change in behavior, as indicated by the device data for a particular client device, is (a) a permanent change in the network communication behavior or (b) an anomaly. In the event that the change in behavior is an anomaly, then the original behavior of the particular client device is expected to resume after a temporary time period.

A data repository associated with the anomaly detection engine 510 stores a candidate set of event types 512. If the event detection engine 506 detects an event 508 that may have caused a change in behavior, as indicated by a particular set of device data, then the anomaly detection engine 510 determines an event type 512 of the event 508. Based on the event type 512 of the event 508, the anomaly detection engine 510 determines whether to classify the particular set of device data into the included dataset 514 or the excluded dataset 516. In some embodiments, the event detection engine 506 detects a combination and/or sequence of events 508 that may have caused a change in behavior, as indicated by a particular set of device data. The anomaly detection engine 510 determines one or more event types 512 of the events. Based on the event types 512, the anomaly detection engine 510 determines whether to classify the particular set of device data into the included dataset 514 or the excluded dataset 516. Additionally or alternatively, the anomaly detection engine 510 classifies the particular set of device data into the included dataset 514 or the excluded dataset 516 based on other factors. Examples of operations for determining whether to include or exclude device data are described below with reference to FIGS. 6-7.

In one or more embodiments, an included dataset 514 is a subset of the raw dataset 504 that is used to determine a network communication configuration 112 of a target device. The included dataset 514 may but does not necessarily include a set of device data corresponding to the target device itself. The included dataset 514 may be used to determine a network communication configuration 112 via various methods. In an embodiment, the included dataset 514 is used as the global dataset 212 illustrated in FIGS. 2-3. The included dataset 514 is used to determine the clusters 322 a-f, which are used to determine a target device profile 104 for a target device. In other embodiments, the included dataset 514 is used to directly as input into the configuration engine 108 illustrated in FIG. 1.

In one or more embodiments, an excluded dataset 516 is a subset of the raw dataset 504 that is not used to determine a network communication configuration 112 of a target device. The excluded dataset 516 is not included in the global dataset 212.

As illustrated, for example, the raw dataset 504 includes device data 204 c, device data 204 d, and device data 204 e. Based on determinations made by the anomaly detection engine 510, the included dataset 514 includes device data 204 c and device data 204 d. The excluded dataset 516 includes device data 204 e.

3. Obtaining a Target Device Profile

FIG. 6 illustrates an example set of operations for obtaining a target device profile, in accordance with one or more embodiments. Obtaining a target device profile includes: (a) identifying correlations between device attributes and behaviors based on analysis of a global dataset; (b) identifying correlations between device attributes and behaviors based on analysis of device type datasets; and (c) determining expected behaviors for a target device based on the global dataset and the device type datasets. One or more operations illustrated in FIG. 6 may be modified, rearranged, or omitted all together. Accordingly, the particular sequence of operations illustrated in FIG. 6 should not be construed as limiting the scope of one or more embodiments.

One or more embodiments include monitoring device data, for devices in a communication network, to obtain a global dataset (Operation 602). A device monitoring engine 502 monitors device data for devices in a communication network. The device monitoring engine 502 may but is not necessarily executed on a network device (such as, an AP). The devices being monitored may but are not necessarily associated with the network device executing the device monitoring engine 502.

The device monitoring engine 502 collects information regarding the device attributes of each device. As an example, a device monitoring engine may intercept data being transmitted from a particular device. The transmitted data may include a media access control (MAC) address of the particular device, for example, 00-CD-FE-C4-12-34. The IEEE may maintain a registry of manufacturers and corresponding blocks of MAC addresses. The block of MAC addresses beginning with the prefix 00-CD-FE may be assigned to Apple, Inc. Since the MAC address of the particular device is within the block of MAC addresses assigned to Apple, Inc., the device monitoring engine may determine that the manufacturer of the particular device is Apple, Inc. The device monitoring device may store “Apple, Inc.” as a device attribute of the particular device. Additionally or alternatively, the device monitoring engine may identify a specific device type of the particular device. Various methods may be used for identifying the device type, such as Dynamic Host Configuration Protocol (DHCP) fingerprinting, Bonjour/mDNS detection or snooping other protocols (such as network discovery, advertisement or management traffic). The device monitoring engine may determine that the particular device is, for example, an iPhone 6s. The device monitoring device may store “iPhone 6s” as a device attribute of the particular device.

As another example, a device monitoring engine may intercept data being transmitted from a particular device. The device monitoring engine may determine that the data is transmitted via channel 11 of the 2.4 GHz band under the IEEE 802.11b/g/n standard. The device monitoring engine may store “channel 11” as a device attribute of the particular device.

As another example, a device monitoring engine may query a particular device for the location of the particular device. Based on a global positioning system (GPS) coupled to the particular device, the particular device may determine that the longitudinal-latitudinal coordinates are 37.77 degrees North and 122.42 degrees East. The particular device may transmit a response to the device monitoring, indicating that the location of the particular device is 37.77 degrees North and 122.42 degrees East. The device monitoring engine may store “37.77 degrees North and 122.42 degrees East” as a device attribute of the particular device.

As another example, a particular device may be within a wireless range of three APs. The particular device may transmit a wireless signal. Each AP may receive the wireless signal at a respective signal strength. Based on the signal strength received by each AP, a device monitoring engine may determine a location of the particular device with respect to the three APs. The device monitoring engine may store the location as a device attribute of the particular device. Additional and/or alternative methods may be used in determining a location of a particular device, such as, using integrated Bluetooth radio.

The device monitoring engine 502 collects information regarding the behaviors of each device. As an example, a device monitoring engine may detect a probe request transmitted by a particular device. The device monitoring engine may detect an authentication request transmitted by the particular device. The device monitoring engine may determine a duration of time between the probe request and the authentication request. The device monitoring engine may store the duration of time between the probe request and the authentication request as a behavior of the particular device.

As another example, a device monitoring engine may detect a roaming request transmitted by a particular device. The device monitoring engine may determine a signal strength of a wireless signal, received by the particular device, at the time the roaming request is detected. The device monitoring engine may store the received signal strength, at the time a roaming request is detected, as a behavior of the particular device.

As another example, a configuration engine may transmit a command to steer a particular device from a first radio to a second radio. A device monitoring engine may detect the command transmitted by the configuration engine. The device monitoring engine may determine whether the particular device successfully associated with the second radio. The device monitoring engine may determine a duration of time needed to switch from associating from the first radio to the second radio. The device monitoring engine may store the information related to steering as behaviors of the particular device. The client device may exhibit different behaviors depending on the specific steering performed. For example, the client device may exhibit different behaviors for (a) steering from the 2.4 GHz band to the 5 GHz band, (b) steering from the 5 GHz band to the 2.4 GHz band, (c) steering within the 2.4 GHz band, (d) steering within the 5 GHz band, (e) steering within an upper band of the 5 GHz band, and/or other types of steering.

The device monitoring engine 502 stores the device attributes and behaviors of a particular device as a particular set of device data. The device monitoring engine 502 stores a set of device data for each monitored device. The sets of device data are referred to herein as a “raw dataset.”

In an embodiment, the device monitoring engine 502 uses the raw dataset as a global dataset. In another embodiment, the device monitoring engine 502 inputs the raw dataset into an anomaly detection engine 510. The anomaly detection engine 510 divides the raw dataset into an included dataset and an excluded dataset. The included dataset is used as the global dataset. Further examples of operations for determining whether to include or exclude device data are described below with reference to FIG. 7.

3.A. Identifying Correlations Between Device Attributes and Behaviors Based on Analysis of a Global Dataset

One or more embodiments include applying an unsupervised learning algorithm to the global dataset to obtain a set of clusters for all device types, each cluster indicating a correlation between a set of device attributes and a set of behaviors (Operation 604). An unsupervised machine learning algorithm 314 b is applied to the global dataset to identify clusters within the global dataset.

As an example, a global dataset may include the following sets of device data:

-   -   (a) Attributes: Channel 44, Wi-Fi Radio Type A—Behaviors:         connection time 35 ms     -   (b) Attributes: Channel 44, Wi-Fi Radio Type B—Behaviors:         connection time 45 ms     -   (c) Attributes: Channel 149, Wi-Fi Radio Type C—Behaviors:         connection time 75 ms

An unsupervised machine learning algorithm may be applied to the global dataset to identify clusters. Since device data including Channel 44 as an attribute is associated with a similar connection time, but device data not including Channel 44 as an attribute is associated with dissimilar connection, a particular cluster may be associated with the attribute Channel 44. The behavior associated with the particular cluster may be the average connection time of all device data within the particular cluster. The average connection time may be 40 ms. Hence, the particular cluster may be as follows: “Attributes: Channel 44—Behaviors: connection time 40 ms.”

3.B. Identifying Correlations Between Device Attributes and Behaviors Based on Analysis of Device Type Datasets

One or more embodiments include applying a classifier function to divide the global dataset into device type datasets (Operation 606). A classifier function 216 may be applied to the global dataset. The classifier function 216 is obtained through application of a supervised machine learning algorithm 214 to a labeled global dataset 202, as described above with reference to FIG. 2.

In an embodiment, the device attributes and/or behaviors associated with a set of device data, within the global dataset, are identified. Scaling may be applied to one or more device attributes and/or behaviors. The classifier function 216 specifies a weight corresponding to each scaled device attribute and/or behavior. A sum of the weighted device attributes and/or behaviors is computed. The classifier function 216 is further associated with mappings between ranges of the weighted sum and device types. If the weighted sum is within a particular range, then the set of device data is associated with the device type mapped to the particular range. Based on the weighted sum, the set of device data is classified into a particular device type dataset.

As an example, a particular set of device data, within a global dataset, may be: “Attributes: Channel 36 (at 40 MHz width), SNR 20 dB—Behaviors: roaming time 25 ms.”

A classifier function may specify that the channel attribute is associated with a weight of 0.5, the SNR attribute is associated with a weight of 0.3, and the roaming time is associated with a weight of 0.2. A weighted sum may be computed by normalizing the channel, SNR, and roaming time values and multiplying them with the corresponding weights. The weighted sum may indicate that a device type associated with the particular set of device data is iPhone 7. The classifier function may classify the particular set of device data into a device type dataset associated with iPhone 7.

One or more embodiments include applying the unsupervised learning algorithm to each device type dataset to obtain a set of clusters for each device type, each cluster indicating a correlation between a set of device attributes and a set of behaviors (Operation 608). An unsupervised machine learning algorithm 314 b is applied to each device type dataset to identify clusters within each device type dataset. Further descriptions regarding the application of an unsupervised machine learning algorithm are included above with reference to Operation 604. The unsupervised machine learning algorithm 314 a and the unsupervised machine learning algorithm 314 b may be the same as or different from each other.

3.C. Determining Expected Behaviors for a Target Device Based on the Global Dataset and the Device Type Datasets

One or more embodiments include determining device attributes for a target device (Operation 610). A device monitoring engine 502 monitors device data corresponding to the target device. Further descriptions regarding monitoring device data are included above with reference to Operation 602. The device monitoring engine 502 monitoring device data corresponding to the target device may but is not necessarily the same as the device monitoring engine 502 performing Operation 602.

One or more embodiments include identifying clusters that share at least one device attribute with the target device (also referred to as “relevant clusters”) (Operation 612). A profile builder 406 (and/or a cluster selection module 408 thereof) identifies the clusters obtained from Operation 604 and Operation 608. The profile builder 406 compares (a) device attributes associated with each cluster and (b) the device attributes of the target device. The profile builder 406 identifies clusters that share at least one device attribute with the target device as relevant clusters.

The device data, obtained by the device monitoring engine 502, may but does not necessarily include a device type of the target device. If the device data, obtained by the device monitoring engine 502, does not include the device type of the target device, then the profile builder 406 may optionally apply a classifier function 216 to the device data of the target device. The classifier function 216 determines a device type associated with the device data of the target device. If a particular device type of the target device is known, using the device monitoring engine 502 and/or the classifier function 216, then the profile builder 406 selects only clusters that are associated with the particular device type.

In an embodiment, the profile builder 406 identifies relevant clusters in at least two phases. In the first phase, the profile builder 406 selects cluster groups 320 a-c that are relevant to the target device. As described above, a cluster group includes clusters obtained from the global dataset. Alternatively, a cluster group includes clusters obtained from a single device type dataset.

The profile builder 406 selects the cluster group obtained from the global dataset as a relevant cluster group. Additionally or alternatively, the profile builder 406 selects at least a subset of the cluster groups obtained from a device type dataset as relevant cluster groups. The selection of relevant cluster groups, from the cluster groups obtained from a device type dataset, is based on the attributes that are known for the target device. If the known attributes of the target device include the device type of the target device, then the profile builder 406 selects only one cluster group, from the cluster groups obtained from a device type dataset. The selected cluster group is the cluster group obtained from the device type dataset associated with the device type of the target device. Alternatively, if the known attributes of the target device does not include the device type of the target device, then the profile builder 406 attempts to select one or more cluster groups, from the cluster groups obtained from a device type dataset, based on other device attributes. As an example, a known attribute of a target device may include a manufacturer of the target device. The manufacturer of the target device may be Apple, Inc. Based on the manufacturer of the target device, a profile builder selects only the cluster groups associated with Apple-manufactured device types, such as iPhone 7, and iPhone 6. The profile builder does not select cluster groups associated with device types of other manufacturers, such as Samsung Galaxy S6, and LG V20.

In the second phase, the profile builder 406 analyzes each selected cluster group to identify clusters that share at least one device attribute with the target device. Clusters that share at least one device attribute with the target device are referred to as “relevant clusters.” The profile builder 406 does not analyze any non-selected cluster groups for relevant clusters. The relevant clusters may include (a) clusters obtained from one or more device type datasets and/or (b) clusters obtained from the global dataset.

One or more embodiments include determining whether the relevant clusters are associated with any common behaviors (Operation 614). The profile builder 406 (and/or a merger module 412 thereof) compares the behaviors associated each relevant cluster to the behaviors associated other relevant clusters. The profile builder 406 identifies any relevant clusters that are associated with the same behavior.

As an example, a target device may be an iPhone 7 operating on channel 11. One relevant cluster may be: “Attributes: Channel 11—Behaviors: correctly steers to a radio determined by the intelligent client steering algorithm, using a specific steering method.” Another relevant cluster may be: “Attributes: iPhone 7—Behaviors: does not correctly steer to a radio determined by the intelligent client steering algorithm, using a specific steering method.” A profile builder may determine that the two relevant clusters are associated with the same behavior, that is, response to the intelligent client steering algorithm. The response to the intelligent client steering algorithm is a common behavior to multiple relevant clusters (even though the value for the behavior is different for the different clusters, one being “correctly steers” and the other being “does not correctly steer”).

In one or more embodiments, for each behavior that is common to multiple relevant clusters, the profile builder 406 applies a merger algorithm to determine an expected value for the behavior for the target device (Operation 616). The profile builder 406 (and/or a merger module 412 thereof) determines a first value, for the common behavior, associated with one relevant cluster. The profile builder 406 determines a second value, for the common behavior, associated with another relevant cluster. The profile builder 406 computes a weighted average of the first value and the second value. The weighted average is an expected value, for the common behavior, for the target device. The weight to be applied to each of the first value and the second value is determined based on various factors, such as:

-   -   (a) an attribute type of the attribute that is shared between a         particular relevant cluster and the target device;     -   (b) a behavior type of the common behavior shared across the         relevant clusters;     -   (c) a number of device attributes that are shared between a         particular relevant cluster and the target device;     -   (d) a number of devices in a particular relevant cluster;     -   (e) a particular time period in which the device data         corresponding to a particular cluster was collected; and     -   (f) a correlation strength between the attributes and the         behaviors associated with a particular cluster.     -   Further examples for applying a merger algorithm to determine an         expected value, for a behavior, for a target device are         described in the related U.S. patent application Ser. No.         15/428,470, filed Feb. 9, 2017, which is hereby incorporated by         reference in its entirety.

One or more embodiments include aggregating behaviors from the relevant clusters to obtain a profile for the target device (Operation 618). The profile builder 406 (and/or an aggregation module 410 thereof) compares the behaviors associated each relevant cluster to the behaviors associated other relevant clusters.

For behaviors that are common to multiple relevant clusters, the profile builder 406 determines an expected value, for the common behavior, as described above with reference to Operation 616. The profile builder 406 includes the expected value, for the common behavior, into the target device profile.

For behaviors that are unique to any single relevant cluster, the profile builder 406 identifies the value, for the behavior, as indicated by the relevant cluster. The profile builder 406 includes the value, for the behavior, into the target device profile.

As an example, a target device may be an iPhone 7 operating on channel 36 (at 40 MHz width). One relevant cluster may be: “Attributes: Channel 36 (at 40 MHz width)—Behaviors: correctly steers to a radio determined by the intelligent client steering algorithm, using a specific steering method.” A profile builder may determine that the behavior, response to the steering algorithm, is not associated with any other relevant clusters.

Additionally, another relevant cluster may be: “Attributes: iPhone 7—Behaviors: connection time 35 ms.” The profile builder may determine that the behavior, connection time, is not associated with any other relevant clusters.

The profile builder may aggregate behaviors for (a) the response to a steering algorithm and (b) the connection time into a target device profile. The target device profile may include the following expected behaviors for the target device: correctly steers to a radio determined by a steering algorithm; and connection time 35 ms.

4. Determining Whether to Include or Exclude Device Data for Configuring a Target Device

FIG. 7 illustrates an example set of operations for determining whether to include or exclude device data for determining a network communication configuration for a target device, in accordance with one or more embodiments. Operations described with reference to FIG. 7 may be incorporated into Operation 602 of FIG. 6. Additionally or alternatively, operations described with reference to FIG. 7 may be used in other ways to determine a network communication configuration for a target device. One or more operations illustrated in FIG. 7 may be modified, rearranged, or omitted all together. Accordingly, the particular sequence of operations illustrated in FIG. 7 should not be construed as limiting the scope of one or more embodiments.

One or more embodiments include monitoring device data for devices during a particular time period (Operation 702). A device monitoring engine 502 monitors device data for devices in a communication network. Further descriptions regarding monitoring device data are included above with reference to Operation 602.

One or more embodiments include determining whether a set of device data indicates a change in network communication behavior in a device (Operation 704). An anomaly detection engine 510 analyzes each set of device data over a particular time period. The anomaly detection engine 510 determines whether a difference in the value of any behavior, in a set of device data during the particular time period, is greater than a threshold. If the difference is greater than the threshold, then the anomaly detection engine 510 determines that there is a change in network communication behavior in the device associated with the set of device data. The device exhibiting the change in network communication behavior may but is not necessarily a target device for which a network communication configuration is being determined.

As an example, a specified threshold for determining a change to the connection time may be 5 ms over a 5-minute time period. An anomaly detection engine may determine that a connection time for a particular device is 30 ms at 10:00 a.m. The anomaly detection engine may determine that the connection time for the particular device is 45 ms at 10:05 a.m. The anomaly detection engine may determine that a difference in the connection time is 15 ms. The anomaly detection engine may determine that the difference in the connection time of 15 ms is greater than the specified threshold of 5 ms. The anomaly detection engine may determine that there is a change in network communication behavior in the particular device.

If a set of device data indicates a change in network communication behavior, then the anomaly detection engine 510 determines whether the change is a permanent change or a temporary anomaly. The anomaly detection engine 510 determines whether the change is permanent based on (a) an event type of an event that may have caused the change in behavior; (b) a number of devices exhibiting the change in the same behavior; and/or (c) the time period during which the change in behavior is observed, as described below with reference to Operations 706-714. Additional and/or alternative factors may be used for determining whether the change in behavior is permanent.

4.A. Determining Whether a Change in Behavior is Permanent Based on an Event Type of an Event that May have Caused the Change

One or more embodiments include determining whether there are any events that may have caused the change in behavior (Operation 706). An event detection engine 506 monitors events associated with the communication network and/or the devices thereof. The event may occur during the same time period from which the device data was monitored at Operation 702. Additionally or alternatively, the event may occur prior to the time period from which the device data was monitored at Operation 702. The event detection engine 506 may obtain information regarding whether any events have occurred in various ways.

In an embodiment, the event detection engine 506 obtains messages from product manufacturers regarding updates to hardware and/or software products. The event detection engine 506 identifies the update as a change associated with one or more devices in the communication network.

In an embodiment, the event detection engine 506 monitors the Internet for messages regarding updates to hardware and/or software products. The event detection engine 506 identifies the update as a change associated with one or more devices in the communication network.

In an embodiment, the event detection engine 506 monitors for manual changes to the communication network and/or the devices thereof. The event detection engine 506 may obtain information regarding manual changes based on user input entered via a user interface. Additionally or alternatively, the event detection engine 506 may obtain information regarding manual changes by monitoring settings of the communication network and/or the devices thereof, and/or packets and/or frames transmitted by network devices and/or client devices. Examples of manual changes include addition and/or removal of a network device (such as, an AP); relocation of a network device; a manual reset of any device that manages, facilitates, and/or supports the communication network (such as, an AP, and an authentication server); addition and/or removal of a client device; relocation of a client device; a manual reset of a client device; a change in the user registered to and/or logged onto a client device.

In an embodiment, the event detection engine 506 monitors for external changes to the communication network. Examples of external changes to the communication network include a change in the interference level, and a change in the noise level.

In an embodiment, the event detection engine 506 monitors for physical changes in the physical environment associated with the communication network. The event detection engine 506 may obtain information regarding physical changes based on user input entered via a user interface. Additionally or alternatively, the event detection engine 506 may obtain information regarding physical changes based on the reflection and/or interference associated with wireless signals in an RF environment. Examples of physical changes include an addition of a table, a removal of a wall or pillar, and a relocation of a shelf.

In an embodiment, the event detection engine 506 determines whether any events, related specifically to the device type associated with the set of device data, may have caused the change in behavior indicated in the set of device data. If the known attributes included in the set of device data do not include the device type, then the anomaly detection engine applies a classifier function the set of device data to determine the device type. The anomaly detection engine then determines whether any events, associated with the device type, have occurred. Additionally or alternatively, the anomaly detection engine then determines whether any combination and/or sequence of events, associated with the device type, has occurred. As an example, an anomaly detection engine may apply a classifier function to a set of device data to determine that the device type, associated with the set of device data, is Apple iPhone 7. The anomaly detection engine may determine whether any events that may have caused a change to an iPhone 7 have occurred. Events that may have caused a change to an iPhone 7 include, for example, an update to an iOS operating system. Events that may have caused a change to an iPhone 7 do not include, for example, an update to an Android operating system.

One or more embodiments include determining an event type of the event that may have caused the change in behavior (Operation 708). The anomaly detection engine 510 determines one or more event types of the one or more events detected at Operation 706. The anomaly detection engine 510 may determine an event type using various methods.

The anomaly detection engine 510 may obtain a set of mappings between events and event types from a data repository. The anomaly detection engine 510 may determine the event type of the event detected at Operation 706 using the set of mappings.

Additionally or alternatively, the anomaly detection engine 510 may use a classifier function to determine an event type of the event detected at Operation 706. The classifier function may be determined using a supervised machine learning algorithm.

Additional and/or alternative methods for determining the event type may be used.

One or more embodiments include determining whether the set of device data should be used, to determine a communication network configuration for a target device, based on the event type determined at Operation 708 (Operation 710). The anomaly detection engine 510 obtains a list of event types from a data repository. The list of event types indicates whether each event type is associated with a permanent change to a communication network behavior or a temporary anomaly in the communication network behavior. The list of event types may be determined based on user input entered via a user interface. Additionally or alternatively, the list of event types may be determined based on machine learning and/or another application.

As an example, a list of event types, stored at a data repository, may include: (a) an update to a software application; (b) a manual reset of a network device; and (c) a change in the noise level in an RF environment. The list of event types may indicate that an update to a software application is a permanent change. The list of event types may indicate that a manual reset of a network device is an anomaly. The list of event types may indicate that the change in the noise level is an anomaly. An anomaly detection engine may determine an event type of a particular event based on the list of event types.

If the event type determined at Operation 708 is associated with a permanent change, then the anomaly detection engine 510 uses the set of device data for determining a network communication configuration for the target device (Operation 716). The anomaly detection engine 510 classifies the set of device data into an included dataset. As described above, the included dataset may be used as the global dataset described above with reference to Operation 602. Additionally or alternatively, the included dataset may be used in other ways to determine a configuration for the target device.

If the event type determined at Operation 708 is associated with an anomaly, then the anomaly detection engine 510 determines whether there are other factors indicating that the change in behavior is permanent, as described below.

4.B. Determining Whether a Change in Behavior is Permanent Based on the Number of Devices Exhibiting the Change in Behavior

One or more embodiments include determining whether the number of devices associated with the change in behavior exceeds a specified threshold (Operation 712). The anomaly detection engine 510 determines whether multiple sets of device data indicate a change in a particular network communication behavior over the same particular time period. Further descriptions relating to determining whether a set of device data indicates a change in behavior are included above with reference to Operation 704. The anomaly detection engine 510 determines whether the number of sets of device data, indicating a change in the particular network communication behavior, exceeds the specified threshold. If the number of sets of device data, indicating a change in the particular network communication behavior, exceeds the specified threshold, then the anomaly detection engine 510 determines that the change in the particular network communication behavior is permanent. Different thresholds may be specified for different behaviors.

As an example, a specified threshold for determining a change to the connection time may be 5 ms over a 5-minute time period. An anomaly detection engine may obtain the following sets of device data at 10:00 a.m.:

(a) Device A: Connection time 30 ms;

(b) Device B: Connection time 38 ms;

(c) Device C: Connection time 50 ms.

The anomaly detection engine may obtain the following sets of device data at 10:05 a.m.:

(a) Device A: Connection time 40 ms;

(b) Device B: Connection time 53 ms;

(c) Device C: Connection time 66 ms.

The anomaly detection engine may determine that the difference in connection time for Device A is 10 ms, which is above the specified threshold of 5 ms. The anomaly detection engine may determine that the difference in connection time for Device B is 15 ms, which is above the specified threshold of 5 ms. The anomaly detection engine may determine that the difference in connection time for Device C is 16 ms, which is above the specified threshold of 5 ms.

Continuing the example, a specified threshold for determining a permanent change in a particular behavior may require that at least three devices exhibit the particular behavior over the same time period. The anomaly detection engine may determine that each of Device A, Device B, and Device C exhibits a change in the connection time between 10:00 a.m. and 10:05 a.m. Since at least three devices exhibit a change in the same behavior, the anomaly detection engine may determine that the change in the connection time is permanent.

If the change in behavior is permanent, then the anomaly detection engine 510 uses the sets of device data for determining a network communication configuration for the target device (Operation 716). The anomaly detection engine 510 classifies the sets of device data into an included dataset. As described above, the included dataset may be used as the global dataset described above with reference to Operation 602. Additionally or alternatively, the included dataset may be used in other ways to determine a configuration for the target device.

If the number of devices exhibiting the change in behavior does not indicate that the change is permanent, then the anomaly detection engine 510 determines whether there are other factors indicating that the change in behavior is permanent, as described below.

4. C. Determining Whether a Change in Behavior is Permanent Based on a Time Period During which the Change in Behavior is Observed

One or more embodiments include determining whether the time period during which the change in behavior is observed exceeds a specified threshold (Operation 714). The anomaly detection engine 510 determines whether one or more sets of device data indicate a change in behavior. Further descriptions relating to determining whether a set of device data indicates a change in behavior are included above with reference to Operation 704. The anomaly detection engine 510 determines whether the change in behavior is observed in the sets of device data for a minimum time period. If the change in behavior is observed in the sets of device data for the minimum time period, then the anomaly detection engine 510 determines that the change in behavior is permanent. Different thresholds may be specified for different behaviors.

As an example, a specified threshold for determining a permanent change to the connection time may be 5 ms over a 10-minute time period. An anomaly detection engine may obtain the following connection times, for a particular device, over a particular time period:

10:00 am: 30 ms

10:02 am: 36 ms

10:04 am: 37 ms

10:06 am: 36 ms

10:08 am: 38 ms

10:10 am: 38 ms

10:12 am: 39 ms

The anomaly detection engine may determine that an initial connection time, at 10:00 am, is 30 ms. The anomaly detection engine may determine that the difference in the connection time, from the initial connection time, is greater than 5 ms for at least a 10-minute time period. The anomaly detection engine may determine that the change in the connection time is permanent.

If the change in behavior is permanent, then the anomaly detection engine 510 uses the sets of device data for determining a network communication configuration for the target device (Operation 716). The anomaly detection engine 510 classifies the sets of device data into an included dataset. As described above, the included dataset may be used as the global dataset described above with reference to Operation 602. Additionally or alternatively, the included dataset may be used in other ways to determine a configuration for the target device.

If the change is not permanent, then the anomaly detection engine 510 refrains from using the one or more sets of device data that indicate the change in behavior (Operation 718). The sets of device data are not used for determining a network communication configuration for the target device. The anomaly detection engine 510 classifies the sets of device data into an excluded dataset.

5. Example Embodiment

A detailed example is described below for purposes of clarity. Components and/or operations described below should be understood as one specific example which may not be applicable to certain embodiments. Accordingly, components and/or operations described below should not be construed as limiting the scope of any of the claims.

A communication network includes the following client devices: Device #01, Device #02, Device #03, and Device #04. Device Data #01 corresponds to Device #01. Device Data #02 corresponds to Device #02. Device Data #03 corresponds to Device #03. Device Data #04 corresponds to Device #04.

A known attribute included Device Data #03 is that a device type of Device #03 is Samsung Galaxy S6. An anomaly detection engine determines that Device Data #03 indicates a change in behavior in Device #03. An event detection engine determines that an update was released for the Android operating system, updating the Android operating system from the Marshmallow version to the Nougat version. The anomaly detection engine determines that an event type corresponding to the event is “update to the operating system.” The anomaly detection engine determines that the event type corresponds to a permanent change. The anomaly detection engine determines that Device Data #03 should be used in determining a network communication configuration for a target device.

The anomaly detection engine determines that Device Data #04 indicates a change in behavior in Device #04. An event detection engine determines that a manual reset of Device Data #04 occurred within a minimum time period prior to the change in behavior in Device #04. The anomaly detection engine determines that an event type corresponding to the event is “manual reset to device.” The anomaly detection engine determines that the event type corresponds to an anomaly. The anomaly detection engine determines that Device Data #04 should not be used in determining a network communication configuration for a target device.

The anomaly detection engine forms a global dataset including Device Data #01, Device Data #02, and Device Data #03, but not Device Data #04.

An unsupervised learning algorithm is applied to a global dataset including Device Data #01, Device Data #02, and Device Data #03. A cluster group, obtained from the global dataset, includes the following clusters:

-   -   (a) Attributes: Aggregation size 2, Network “Michael's         Home”—Behaviors: connection time 36 ms; and     -   (b) Attributes: Aggregation size 2, Channel 36 (at 40 MHz         width)—Behaviors: roaming time 35 ms.

A classifier function is applied to the global dataset. The classifier function determines that Device Data #01 and Device Data #02 are associated with the device type, Apple iPhone 7. The classifier function determines that Device Data #03 is associated with the device type, Samsung Galaxy S6.

The unsupervised learning algorithm is applied to the device type dataset associated with iPhone 7, including Device Data #01, and Device Data #02. A cluster group, obtained from the iPhone 7 dataset, includes the following clusters:

-   -   (a) Attributes: iPhone 7, Channel 36 (at 40 MHz         width)—Behaviors: connection time 30 ms.

The unsupervised learning algorithm is applied to the device type dataset associated with Galaxy S6, including Device Data #03. A cluster group, obtained from the Galaxy S6 dataset, includes the following clusters:

-   -   (a) Attributes: Galaxy S6, Channel 36 (at 40 MHz         width)—Behaviors: enters power-save mode if not used for 1         minute.

A target device is associated with the following attributes: iPhone 7, and Channel 36 (at 40 MHz width). A profile builder identifies cluster groups relevant to the target device. The relevant cluster groups include: (a) the cluster group obtained from the global dataset, and (b) the cluster group obtained from the iPhone 7 dataset. The relevant cluster groups do not include the cluster group obtained from the Galaxy S6 dataset.

The profile builder analyzes the cluster group obtained from the global dataset for clusters sharing a device attribute with the target device. A first cluster, “Attributes: Aggregation size 2, Network ‘Michael's Home’—Behaviors: connection time 36 ms,” does not share any attributes with the target device. The first cluster is not selected. The second cluster, “Attributes: Aggregation size 2, Channel 36 (at 40 MHz width)—Behaviors: roaming time 35 ms,” shares an attribute with the target device. The second cluster is selected as a relevant cluster.

The profile builder analyzes the cluster group obtained from the iPhone 7 dataset for clusters sharing a device attribute with the target device. The cluster, “iPhone 7, Channel 36 (at 40 MHz)—Behaviors: connection time 30 ms,” shares an attribute with the target device. The cluster is selected as a relevant cluster.

The profile builder aggregates the behaviors, from the relevant clusters, into a target device profile. The target device profile includes the following expected behaviors for the target device: (a) roaming time 35 ms; and (b) connection time 30 ms.

6. Miscellaneous; Extensions

Embodiments are directed to a system with one or more devices that include a hardware processor and that are configured to perform any of the operations described herein and/or recited in any of the claims below.

In an embodiment, a non-transitory computer readable storage medium comprises instructions which, when executed by one or more hardware processors, causes performance of any of the operations described herein and/or recited in any of the claims.

Any combination of the features and functionalities described herein may be used in accordance with one or more embodiments. In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the invention, and what is intended by the applicants to be the scope of the invention, is the literal and equivalent scope of the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction.

7. Hardware Overview

According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or network processing units (NPUs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, FPGAs, or NPUs with custom programming to accomplish the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques.

For example, FIG. 8 is a block diagram that illustrates a computer system 800 upon which an embodiment of the invention may be implemented. Computer system 800 includes a bus 802 or other communication mechanism for communicating information, and a hardware processor 804 coupled with bus 802 for processing information. Hardware processor 804 may be, for example, a general purpose microprocessor.

Computer system 800 also includes a main memory 806, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 802 for storing information and instructions to be executed by processor 804. Main memory 806 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 804. Such instructions, when stored in non-transitory storage media accessible to processor 804, render computer system 800 into a special-purpose machine that is customized to perform the operations specified in the instructions.

Computer system 800 further includes a read only memory (ROM) 808 or other static storage device coupled to bus 802 for storing static information and instructions for processor 804. A storage device 810, such as a magnetic disk or optical disk, is provided and coupled to bus 802 for storing information and instructions.

Computer system 800 may be coupled via bus 802 to a display 812, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 814, including alphanumeric and other keys, is coupled to bus 802 for communicating information and command selections to processor 804. Another type of user input device is cursor control 816, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 804 and for controlling cursor movement on display 812. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.

Computer system 800 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 800 to be a special-purpose machine. According to one embodiment, the techniques herein are performed by computer system 800 in response to processor 804 executing one or more sequences of one or more instructions contained in main memory 806. Such instructions may be read into main memory 806 from another storage medium, such as storage device 810. Execution of the sequences of instructions contained in main memory 806 causes processor 804 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.

The term “storage media” as used herein refers to any non-transitory media that store data and/or instructions that cause a machine to operate in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 810. Volatile media includes dynamic memory, such as main memory 806. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge, content-addressable memory (CAM), and ternary content-addressable memory (TCAM).

Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 802. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.

Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 804 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 800 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 802. Bus 802 carries the data to main memory 806, from which processor 804 retrieves and executes the instructions. The instructions received by main memory 806 may optionally be stored on storage device 810 either before or after execution by processor 804.

Computer system 800 also includes a communication interface 818 coupled to bus 802. Communication interface 818 provides a two-way data communication coupling to a network link 820 that is connected to a local network 822. For example, communication interface 818 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 818 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 818 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

Network link 820 typically provides data communication through one or more networks to other data devices. For example, network link 820 may provide a connection through local network 822 to a host computer 824 or to data equipment operated by an Internet Service Provider (ISP) 826. ISP 826 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 828. Local network 822 and Internet 828 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 820 and through communication interface 818, which carry the digital data to and from computer system 800, are example forms of transmission media.

Computer system 800 can send messages and receive data, including program code, through the network(s), network link 820 and communication interface 818. In the Internet example, a server 830 might transmit a requested code for an application program through Internet 828, ISP 826, local network 822 and communication interface 818.

The received code may be executed by processor 804 as it is received, and/or stored in storage device 810, or other non-volatile storage for later execution.

In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the invention, and what is intended by the applicants to be the scope of the invention, is the literal and equivalent scope of the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. 

What is claimed is:
 1. A non-transitory computer readable medium comprising instructions which, when executed by one or more hardware processors, cause performance of operations comprising: obtaining, during a first time period, a first set of device data associated with a client device; determining that the first set of device data indicates a first change in network communication behavior in the client device during the first time period; determining that a first event that is a first candidate cause of the first change in network communication behavior has occurred; responsive to determining that the first event is associated with a first event type: including the first set of device data into a set of input data used for determining a configuration for the client device; wherein including the first set of device data into the set of input data used for determining the configuration for the client device comprises: using at least the first set of device data for determining a profile for the client device; applying a configuration algorithm to the profile for determining the configuration for the client device; changing the configuration for the client device based on the profile for the client device as determined based on the set of input data including the first set of device data; obtaining, during a second time period, a second set of device data associated with the client device; determining that the second set of device data indicates a second change in network communication behavior in the client device during the second time period; determining that a second event that is a second candidate cause of the second change in network communication behavior has occurred; responsive to determining that the second event is associated with a second event type: excluding the second set of device data from the set of input data used for determining the configuration for the client device; wherein the first event type and the second event type are different; wherein excluding the second set of device data from the set of input data used for determining the configuration for the client device comprises: refraining from using the second set of device data for determining the profile for the client device.
 2. The medium of claim 1, wherein the operations further comprise: subsequent to determining that the second event is associated with the second event type: determining that the second change in network communication behavior in the client device is observed in multiple sets of device data associated with the client device obtained over a particular time period; responsive to determining that the second change in network communication behavior in the client device is observed in the multiple sets of device data associated with the client device obtained over the particular time period: including the second set of device data into the set of input data used for determining the configuration the client device; changing the configuration for the client device based on the set of input data including the second set of device data.
 3. The medium of claim 1, wherein the operations further comprise: subsequent to determining that the second event is associated with the second event type: obtaining a plurality of sets of device data associated with a plurality of client devices; determining that the plurality of sets of device data indicates the second change in network communication behavior in the plurality of client devices; responsive to determining that a number of client devices in the plurality of client devices is above a threshold: including the second set of device data into the set of input data used for determining the configuration the client device; changing the configuration for the client device based on the set of input data including the second set of device data.
 4. The medium of claim 1, further comprising: obtaining, during a third time period, a third set of device data associated with the client device; determining that the third set of device data indicates a third change in network communication behavior in the client device during the third time period; responsive to determining that no event that is a candidate cause of the third change in network communication behavior has occurred: excluding the third set of device data from the set of input data used for determining the configuration for the client device.
 5. The medium of claim 1, wherein the second change in network communication behavior in the client device comprises at least one of: a change in a protocol behavior; a change in a connection behavior; a change in a power-save behavior; a change in a roaming behavior; a change in a response to a particular configuration algorithm; and a change in a response to a particular radio-frequency (RF) environment.
 6. The medium of claim 1, wherein the first event type comprises at least one of: a change to an operating system of the client device; a change to a software application executing on the client device; and a change in a deployment of access points in a RF environment of the client device.
 7. The medium of claim 1, wherein the second event type comprises at least one of: a manual reset of the client device; and a manual reset of an access point in a RF environment of the client device.
 8. The medium of claim 1, wherein the second event occurred during the second time period.
 9. The medium of claim 1, wherein using at least the first set of device data for determining the profile for the client device comprises: based on at least the first set of device data, modifying the profile from indicating a first expected behavior for the client device to indicating a second expected behavior for the client device.
 10. The medium of claim 9, wherein the first expected behavior comprises at least one of: (a) a protocol behavior; (b) a connection behavior; (c) a power-save behavior; (d) a roaming behavior; (e) a response to a particular configuration algorithm; and (f) a response to a particular radio-frequency (RF) environment.
 11. A system, comprising: at least one device including a hardware processor; and the system configured to perform operations comprising: obtaining, during a first time period, a first set of device data associated with a client device; determining that the first set of device data indicates a first change in network communication behavior in the client device during the first time period; determining that a first event that is a first candidate cause of the first change in network communication behavior has occurred; responsive to determining that the first event is associated with a first event type: including the first set of device data into a set of input data used for determining a configuration for the client device; wherein including the first set of device data into the set of input data used for determining the configuration for the client device comprises: using at least the first set of device data for determining a profile for the client device; applying a configuration algorithm to the profile for determining the configuration for the client device; changing the configuration for the client device based on the profile for the client device as determined based on the set of input data including the first set of device data; obtaining, during a second time period, a second set of device data associated with the client device; determining that the second set of device data indicates a second change in network communication behavior in the client device during the second time period; determining that a second event that is a second candidate cause of the second change in network communication behavior has occurred; responsive to determining that the second event is associated with a second event type: excluding the second set of device data from the set of input data used for determining the configuration for the client device; wherein the first event type and the second event type are different; wherein excluding the second set of device data from the set of input data used for determining the configuration for the client device comprises: refraining from using the second set of device data for determining the profile for the client device.
 12. The system of claim 11, wherein the operations further comprise: subsequent to determining that the second event is associated with the second event type: determining that the second change in network communication behavior in the client device is observed in multiple sets of device data associated with the client device obtained over a particular time period; responsive to determining that the second change in network communication behavior in the client device is observed in the multiple sets of device data associated with the client device obtained over the particular time period: including the second set of device data into the set of input data used for determining the configuration the client device; changing the configuration for the client device based on the set of input data including the second set of device data.
 13. The system of claim 11, wherein the operations further comprise: subsequent to determining that the second event is associated with the second event type: obtaining a plurality of sets of device data associated with a plurality of client devices; determining that the plurality of sets of device data indicates the second change in network communication behavior in the plurality of client devices; responsive to determining that a number of client devices in the plurality of client devices is above a threshold: including the second set of device data into the set of input data used for determining the configuration the client device; changing the configuration for the client device based on the set of input data including the second set of device data.
 14. The system of claim 11, further comprising: obtaining, during a third time period, a third set of device data associated with the client device; determining that the third set of device data indicates a third change in network communication behavior in the client device during the third time period; responsive to determining that no event that is a candidate cause of the third change in network communication behavior has occurred: excluding the third set of device data from the set of input data used for determining the configuration for the client device.
 15. The system of claim 11, wherein the second change in network communication behavior in the client device comprises at least one of: a change in a protocol behavior; a change in a connection behavior; a change in a power-save behavior; a change in a roaming behavior; a change in a response to a particular configuration algorithm; and a change in a response to a particular radio-frequency (RF) environment.
 16. The system of claim 11, wherein the first event type comprises at least one of: a change to an operating system of the client device; a change to a software application executing on the client device; and a change in a deployment of access points in a RF environment of the client device.
 17. The system of claim 11, wherein the second event type comprises at least one of: a manual reset of the client device; and a manual reset of an access point in a RF environment of the client device.
 18. The system of claim 11, wherein using at least the first set of device data for determining the profile for the client device comprises: based on at least the first set of device data, modifying the profile from indicating a first expected behavior for the client device to indicating a second expected behavior for the client device.
 19. The system of claim 18, wherein the first expected behavior comprises at least one of: (a) a protocol behavior; (b) a connection behavior; (c) a power-save behavior; (d) a roaming behavior; (e) a response to a particular configuration algorithm; and (f) a response to a particular radio-frequency (RF) environment.
 20. A method, comprising: obtaining, during a first time period, a first set of device data associated with a client device; determining that the first set of device data indicates a first change in network communication behavior in the client device during the first time period; determining that a first event that is a first candidate cause of the first change in network communication behavior has occurred; responsive to determining that the first event is associated with a first event type: including the first set of device data into a set of input data used for determining a configuration for the client device; wherein including the first set of device data into the set of input data used for determining the configuration for the client device comprises: using at least the first set of device data for determining a profile for the client device; applying a configuration algorithm to the profile for determining the configuration for the client device; changing the configuration for the client device based on the profile for the client device as determined based on the set of input data including the first set of device data; obtaining, during a second time period, a second set of device data associated with the client device; determining that the second set of device data indicates a second change in network communication behavior in the client device during the second time period; determining that a second event that is a second candidate cause of the second change in network communication behavior has occurred; responsive to determining that the second event is associated with a second event type: excluding the second set of device data from the set of input data used for determining the configuration for the client device; wherein the first event type and the second event type are different; wherein excluding the second set of device data from the set of input data used for determining the configuration for the client device comprises: refraining from using the second set of device data for determining the profile for the client device; wherein the method is performed by at least one device including a hardware processor. 